Search for: All records

Abstract Understanding how cortical circuits generate complex behavior requires investigating the cell types that comprise them. Functional differences across pyramidal neuron (PyN) types have been observed within cortical areas, but it is not known whether these local differences extend throughout the cortex, nor whether additional differences emerge when larger-scale dynamics are considered. We used genetic and retrograde labeling to target pyramidal tract, intratelencephalic and corticostriatal projection neurons and measured their cortex-wide activity. Each PyN type drove unique neural dynamics, both at the local and cortex-wide scales. Cortical activity and optogenetic inactivation during an auditory decision task revealed distinct functional roles. All PyNs in parietal cortex were recruited during perception of the auditory stimulus, but, surprisingly, pyramidal tract neurons had the largest causal role. In frontal cortex, all PyNs were required for accurate choices but showed distinct choice tuning. Our results reveal that rich, cell-type-specific cortical dynamics shape perceptual decisions. 

Federated Learning (FL) has attracted increasing attention in recent years. A leading training algorithm in FL is local SGD, which updates the model parameter on each worker and averages model parameters across different workers only once in a while. Although it has fewer communication rounds than the classical parallel SGD, local SGD still has large communication overhead in each communication round for large machine learning models, such as deep neural networks. To address this issue, we propose a new communicationefficient distributed SGD method, which can significantly reduce the communication cost by the error-compensated double compression mechanism. Under the non-convex setting, our theoretical results show that our approach has better communication complexity than existing methods and enjoys the same linear speedup regarding the number of workers as the full-precision local SGD. Moreover, we propose a communication-efficient distributed SGD with momentum, which also has better communication complexity than existing methods and enjoys a linear speedup with respect to the number of workers. At last, extensive experiments are conducted to verify the performance of our proposed methods. 

Although the distributed machine learning methods can speed up the training of large deep neural networks, the communication cost has become the non-negligible bottleneck to constrain the performance. To address this challenge, the gradient compression based communication-efficient distributed learning methods were designed to reduce the communication cost, and more recently the local error feedback was incorporated to compensate for the corresponding performance loss. However, in this paper, we will show that a new "gradient mismatch" problem is raised by the local error feedback in centralized distributed training and can lead to degraded performance compared with full-precision training. To solve this critical problem, we propose two novel techniques, 1) step ahead and 2) error averaging, with rigorous theoretical analysis. Both our theoretical and empirical results show that our new methods can handle the "gradient mismatch" problem. The experimental results show that we can even train faster with common gradient compression schemes than both the full-precision training and local error feedback regarding the training epochs and without performance loss.